fix(runtime): carve liveness out of the identity step so a configuration fault cannot restart a pod - #16561

Fixes #15910

A liveness probe that answers 503 for a configuration fault tells the orchestrator to restart a pod whose fault no restart can fix. This carries out the maintainer's ruling on #15910 (comment 5559821649, director seat, decision batch #57, 2026-09-06, maintainer reply verbatim 「同意」), option C:

Ruling. Carve liveness out of the identity step. /health (liveness) answers 200 whenever the process can serve HTTP, regardless of configuration faults; /ready (readiness) keeps returning 503 for the identity/configuration fault so traffic is withheld until the fault is fixed. A configuration fault must never restart a pod that cannot be fixed by restarting.

Option A (leave as shipped) and option B (make the posture read lazy for anonymous requests) are not taken; B was weighed and rejected in the ruling round for changing when a security-relevant value is read for every anonymous request.

The premise, measured on this branch — not inherited

PR #15909 is merged, so the behaviour is main's today. This branch measured it rather than quoting it: with the carve-out ablated out of dispatch() (one line, restored under a trap, byte-verified), the acceptance test reports

AssertionError: expected { health: 503, ready: 503 } to deeply equal { health: 200, ready: 503 }

That left-hand object is origin/main's answer to two uncredentialed probes on one kernel whose tenancy service is registered and fails to build.

/ready really does distinguish running from initializing — confirmed, with the nuance

Triage flagged this as reproduced-by-nobody and inherit-at-your-risk. Read at source (packages/runtime/src/http-dispatcher.ts, the /ready domain registration):

const state: string = typeof host?.getState === 'function' ? host.getState() : 'running';
if (state !== 'running') {
    return { handled: true, response: this.error('Service not ready', 503, { state }) };
}

with KernelState = 'idle' | 'initializing' | 'running' | 'stopping' | 'stopped' (packages/core/src/kernel-base.ts). So the claim holds, and the precise shape is worth stating: the branch is running versus everything else, not an initializing-specific arm — and the state that was read is echoed on the wire in the 503 body's details.state, so an operator reading it sees initializing told apart from stopping and stopped. The suite measures that echo (details.state is idle on an unstarted fixture kernel). /ready is therefore already the route meaning "this deployment is not serving correctly", and it is already wired.

What changed

A route may now declare liveness: true on its DomainRoute registry entry. HttpDispatcher.dispatch() runs such a route's handler directly — no resolveRequestScope, no ADR-0069 auth gate, no project-membership gate, so nothing that reads configuration or credentials is evaluated. /health declares it; its payload (status, timestamp, version, uptime) was already process-local, which is the ruling's first execution note satisfied by construction rather than by promise.

The identity step is the only thing /health loses. The two gates the carve-out also skips were already no-ops for this path, by measurement at source: enforceAuthGate opens with isAuthGateAllowlisted(cleanPath) and ALLOW_SUFFIXES in packages/core/src/security/auth-gate.ts names /health; enforceProjectMembership opens with a skipPaths list that names it too. So skipping them is observationally identical, and the whole behavioural delta of this PR is the one thing the ruling asked for.

Everything else is deliberately untouched:

• /ready keeps the full identity step and its 503 body, so operator dashboards keep their signal (ruling's second execution note).
• The fix(runtime): the tenancy posture seam tells "never registered" from "registered and failed" at the runtime door #15909 re-raise is not softened anywhere else: GET /data/task under the same fault still leaves dispatch() as a 503 SERVICE_UNAVAILABLE.
• POST /health is not liveness (the route declares methods: ['GET'], and the carve-out inherits that).
• The environment-scoped /environments/:id/health keeps today's behaviour end to end — the carve-out sits above the scoped-URL strip on purpose, and no orchestrator wires a scoped probe.

How the liveness route set is DERIVED, not listed

The card raised this to a binding constraint, citing this repo's own drifted hand-maintained lists (#15915, #16169, #15584). There is no new list:

resolveLiveness(path: string, method: string): DomainRoute | undefined {
    const route = this.resolve(path, method);
    return route?.liveness ? route : undefined;
}

The liveness set is a projection of the live route table — resolve() plus one field read — so it answers through the same matcher that picks the handler. It cannot name a route that is not registered, cannot miss one that is, and cannot disagree with the matcher about which route a path reaches (a non-liveness route registered earlier shadows here exactly as it shadows in resolve). A future liveness route becomes liveness by declaring it on the same object that makes the route exist, which is the only edit that cannot be forgotten.

The suite pins the derivation adversarially: it registers a new liveness route through the public registerDomainHandler seam and drives it through the same fault. A hard-coded '/health' test inside dispatch() would fail that leg alone while every other leg stayed green.

Docs (ruling's third execution note)

content/docs/deployment/self-hosting.mdx — the page that already carries the probe table and the reference Kubernetes manifest — gains the paragraph: which probe goes on livenessProbe, which on readinessProbe, never crossed, and what crossing them costs (CrashLoopBackOff in place of the fault). content/docs/deployment/cli.mdx's os serve section gains a short note and a link to it rather than a second copy of the field mapping, so the mapping stays in one place.

Tests

Run at e564d8be (git rev-parse --short HEAD of the commit under measurement).

• New: packages/runtime/src/http-dispatcher.liveness-carve-out.test.ts — 12 tests, all passing. The acceptance test the ruling names asserts the pair in one expression, against one forced fault:

  expect({ health: health.status, ready: ready.status }).toEqual({ health: 200, ready: 503 });

  Either half alone is non-discriminating: /health 200 alone is satisfied by deleting the re-raise outright, /ready 503 alone is satisfied by origin/main. Both probes are driven through the real mounted routes on a real ObjectKernel whose tenancy service factory throws, so the rejection under test is the service registry's own unbranded one.

• The two 503s are told apart rather than assumed: under the fault /ready answers the identity step's declared-5xx envelope (message withheld, no details), while with a healthy tenancy on the same wiring it answers its own verdict (Service not ready, details.state). Without that control the readiness half would prove nothing.

• Ablation, in the direction predicted: reverting the one carve-out lookup turns 4 of the 12 red (the pair, the payload, the dispatch-level leg, and the derived-set leg) and leaves 8 green — the 8 that describe unchanged behaviour. Mutation and restore were both proved on disk (occurrence counts before/after, git hash-object back to the HEAD blob, git diff HEAD empty), under a trap ... EXIT INT TERM. No dist is involved: the suite imports ./http-dispatcher.js relatively, which resolves to source — demonstrated by the fact that the new resolveLiveness leg passes on a tree where @objectstack/runtime was never rebuilt, while the cross-package http-conformance suite could not resolve the package at all until it was.

Gates

Derived from the tree at runtime, no paths passed: node scripts/pm/dispatch-gates.mjs --commands (repo asserted with --repo objectstack-ai/objectstack), each command run byte-for-byte as printed and recorded as it ran. Reconciled against the tool's own count, not against arithmetic over my list:

✓ dispatch-gates --ran: 94 derived famil(ies) accounted for — 94 run, 0 NOT-MEASURED.

93 of the 94 pass. The one that does not is PREREQUISITE NOT MET (exit 3) — nothing measured, neither a pass nor a finding, and it says so itself: check:dual-build-cjs-loads reads built output for 38 packages that have no dist in this worktree. CI builds fresh and measures it.

check:type-check-debt first reported the same exit 3 here, for a different reason — its --re-measure leg exhausted a 4 GB heap on a whole-tree tsc and refused to record a 0. Re-run with --max-old-space-size=8192 it passes: OK — 76/80 workspace packages type-checked, 4 in the DEBT ledger, with tsc itself still under the gate's own pinned 6144 MB CI-shaped ceiling. The first reading was an environment limit, not a ratchet movement.

The separate artifact-roster block (36 families, 46 command spellings) was run on its own, as it sits beside the derived total rather than inside it: 43 pass, and the three that do not are wiring or prerequisite, not verdicts. check:partof-closing-keyword prints NOT WIRED with no PR context and was then run wired — this body plus this PR's commit list — and passes. check:single-claim-paths needs a GitHub token to list the PR's files; retried against this PR number it gets GitHub API 401, so it stays NOT MEASURED here and CI measures it. check:react-declaration-parity has no objectui sdui.manifest.json to compare against, which is by design: it is an on-demand gate triggered by an .objectui-sha bump this PR does not make.

Full per-command results are in the report comment on #15910.

Clause ②

no, re-derived from the delivered diff (the claim's declaration was no; this is the re-derivation the dispatch required, not an inheritance):

• Path limb: zero files under packages/spec/src/**. No *.zod.ts contract schema, no ERROR_CODE_LEDGER / StandardErrorCode entry, no new error code.
• Wire limb: both bodies involved are pre-existing declared shapes and are byte-unchanged. What moved is which declared body one route returns under a fault — a runtime behaviour change, which the criteria place on the explicit negative boundary ("a runtime permission/security behaviour change is not clause ②").
• The additive TypeScript surface (DomainRoute.liveness, DomainHandlerRegistry.resolveLiveness) is API-surface widening on @objectstack/runtime, paid for by the minor changeset — not a published contract schema.

No needs:contract-review label is applied.

Base

Branched at f48f3f1b2 and not merged with origin/main, deliberately and declared rather than skipped: everything above was measured on that base, and the commits that landed since (through ce8caba91) touch 17 files with zero overlap with this PR's 7, and none of them touches the dispatcher's identity step, the domain registry, or probe routing. AGENTS.md scopes the pre-PR re-run to incoming commits that touch the same packages or the same behaviour; these do not. The PR's own CI builds the merge ref, and the merge queue re-verifies on the rebuilt generation, so the landing seat's merge is the right place for it — merging here would only invalidate the readings above against a main that moves every few minutes on this container.

验收备注

Observed while reading, not filed and not fixed here — none is a reproducible defect, a violation of declared contract text, or a metadata-authoring trap:

• Two pre-existing hand-written path lists already name /health: ALLOW_SUFFIXES in packages/core/src/security/auth-gate.ts and skipPaths in HttpDispatcher.enforceProjectMembership. Both cover more than liveness (/auth, /cloud, /ready, /discovery) and are read from more than one door, so neither is subsumed by the derived liveness set and neither was touched. Worth knowing that the liveness carve-out now precedes both for /health.
• packages/runtime/src/route-ledger.ts carries a third place that calls GET /health a liveness probe, in prose on its ledger row. It is ratcheted by route-ledger.conformance.test.ts against the dispatcher's domains, so it is not an unwatched list; left as is.